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L TECHNICAL FIELD OF THE INVENTION 

The present invention relates to a systems for opto-electronic, non-contanct angle measurement 
methods during the process of sheet metal bending by the bending machine using one or more presses 
and where the bending occurs along one line. When the metal sheet is being bent a laser pattern is pro- 
jected onto the surface of the sheet being bent and the displacement and deformation of this pattern in 
the field of view of the camera is used to derive the angle. 

EL BACKGROUND ART 

Non-contact measurement systems for measuring the bending.angle are based on projecting a cer- 
tain light pattern onto the metal surface. The following patents were considered relevant to the applica- 
tion of angle measurement U.S.4,564,765, U.S.5,53 1,087, U.S.5,531,900 and the following patent was 
relevant to the different parts of the used method U.S. 6,02 1,222. 
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In U.S.4*564,765 (expired), the projected pattern is a set of laser spots. A particular configuration 
of the spots and the distance between them is used to compute the angle. The main limitation of the 
method is the speed and precision of point detection. 

In U.S.5,53 1,087 (pending), two lines were projected and their relative rotation was used to com- 
pute the angle. This method unless its original design, presents several drawbacks. First, measuring and 
light emitting devices arc spread over the large area along the bending ram thus obstructing the access 
to the machine for the operator. Second, a change in light emitting device position is required to adapt 
the system for a new angle measurement. Third, pattern measurements are done without taking into 
account perspective distortion of the image. 

In U.S.5, 196,900 (pending), a set of spots (points) is used as the pattern and as mentioned earlier, 
their detection requires exhaustive search in the image and not suitable for fast detection. 

With respect to the methods to estimate the bending angle from patterns with particular shape, a 
circle detection algorithm is suggested in U.S.6,02 1,222. The performance with respect to speed needs, 
however, to be analyzed. 

The herein presented method is suitable for and is efficiently implemented on an image sensor with 
pixel-based access. This selective access allows to speed up the computation and communication pro- 
cesses so that the angle measurement can be performed in just a few milliseconds. Another advantage 
of the suggested method is the versatility that allows many possibilities to position the measuring sys- 
tem on the bending machine. 

m. DISCLOSURE OF THE INVENTION 

The opto-electronic approach to angle measurement consists in projecting a laser pattern onto the 
sheet metal being bent and measuring the motion and deformation of this pattern during bending. The 
measured motion and deformation can then be used to directly compute the angle between two posi- 
tions of the sheet metal. 

The general algorithm of the angle measurement is presented on Fig. 2 and comprises several 
staged explained in the following sections. The stage of rapid localization the laser pattern in the field 
of view of the camera is explained in section HI. 1. The stage for computing displacement and deforma- 
tion of the projected pattern is presented in section 2. Section 3 describes the computation of the angle 
from the properties of the projected pattern. 

Diverse laser patterns composed of several crossing or parallel lines or one or more circles are suit- 
able for measuring the angle. For example, for "the cross laser patternthe information that is used to 
compute the bending angle is the intersection point of two lines of the cross. Detailed angle computa- 
tions from pattern position are shown for cross laser pattern and the parallel lines. 

Realization of the system is described in section V. 
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1. Fast pattern localization 

To rapidly find location of the cross pattern in the field of view, the algorithm described in Fig. 3 is 
used. It is based on a set of horizontal Scan lines S i (47) that are evenly spaced in vertical direction as 
shown in Fig. 4.b,c.d. 

Only one line S ( of the image sensor is acquired at a time and its intensity contents is analyzed to 
find two significant peaks based on intensity maxima (see Fig. 4.a). If two peaks A { ,B i are found, their 
positions are stored and the scanning continues. If no peaks were detected, the next scanline is acquired 
and analyzed. If three pairs of peaks are found, the scanning stops and an ordering procedure is 
invoked. 

Three pairs of points A 1 i B l ; A 2 ,B 2 ; A 3 ,B 3 , are necessary to uniquely find the approximate posi- 
tion of the crossing lines. However, three pairs of points can correspond to one of the three configura- 
tions outlined in Fig. 4(b,c,d). To efficiently recognize which of the three configurations is currenly 
observed, one need to test the collinearity of point triples and decide which point correspond to which 
line. To check the collinearity, the following determinants (indicated by shaded areas in Fig. 5) are to 
be computed first: 

and then their sums need to be computed as follows: 

A^^i^H^^K. (2) 
A 3 = 1^1*2*31 + \ B i A 2 A l\ 

The first configuration (Fig. 4.a) can then be distinguished by the fact that the first sum A] is 
smaller in absolute value than the sum of other two A 2 + A 3 : 

2A l <A 2 + A 3 (3) 

and at the same time its absolute value is limited by a fixed threshold A x < threshold that defines 
the limit of collinearity. 

Similarly, the second and third configurations can be distinguished with the conditions 
2A2 < Aj + A 3 and 2A 3 < Aj + Aj respectively and conditions on the absolute values of Aj and A 3 . Dis- 
tinguishing between configurations allows to split the six points A lf B t ; A 2 ,B 2 ; A 3 t B 3 in two groups 
of three points each, belonging to each line. The three points on each line will thus determine positions 
of both lines in the image. 

This localization operation is very time-efficient since it requires the acquisition and analysis of 
several lines per image only, as well as evaluation of determinants. 
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The similar procedure can be used to locate the pattern of parallel lines as shown in Fig. 6. A set of 
maxima points arc located on the scan lines. Their order is used to group them for defining where the 
laser pattern is located. 

2. Fast computation of precise pattern position 

Once the approximate pattern position was provided by the localization step, we can proceed with 
the estimation of precise pattern position and properties. The precise position of the cross is the inter- 
section of two lines. The precise position of a line can be obtained by fitting the line equation to a num- 
ber of precise points that belong to the line. Those precise positions are obtained by analyzing the 
neighborhood of the approximate line position as shown in Fig. 7. 

The Horizontal short scan lines (22) of adjustable pixels length are acquired along the approximate 
position of each line found previously in the localization step (see Fig. 7.a). These short scanlines are 
acquired from Upper scanning limit (29) to Upper middle scanning limit (31) and from Lower middle 
scanning limit (32) to Lower scanning limit (30). The Upper scanning limit (29) and Lower scanning 
limit (30) are defined by the first and last Scan lines S { (47) that has two significant peaks (see Fig. 4). 
The Lower scanning limit (30) is defined by the last Scan lines S t (47) in Fig. 4 that has two significant 
peaks. The Upper middle scanning limit (3 1) and Lower middle scanning limit (32) are defined to avoid 
acquiring information in the area near the intersection point (45) where interference between two cross- 
ing lines can occur (the found peak can belong to any of two lines). 

The Horizontal short scan lines (22) are acquired in the image and intensity peak Intensity maxi- 
mum in short line (48) inside each of them is detected that correspond to precise position of the laser 
line of the Laser pattern (4) across the short scanlirie as shown in Fig. 7(b). To obtain a higher precision 
than a pixel, a subpixel accuracy method is used for each scan line as in Fig. 7(c). A parabola or gauss- 
ian is used to approximate the maxima pixel (48) and its two adjacent pixels and the coordinate of the 
parabola maximum Precise maxima position (44)' is retained as the precise maximum position for the 
current short scanline. 

The set of computed line points Precise maxima position (44) is used to fit a line Fitted line (43) as 
shown in Fig. 7(d) in a Least Mean Squares (LMS) sense to these points. The obtained two equations of 
two crossing lines are used to analytically find the Precise intersection (45) that gives a precise cross 
position. 

3. Geometry of the angle measurements : 

Precise position, orienation and deformation of the laser pattern can be used to derive the angle of 
the sheet metal orientation. For the cross pattern, the displacement of the cross center can be used to 
compute the angle as described in the following section. For a set of parallel lines their combined orien- 
tation can be converted into the bending angle, y 

4' 

FEUILLE DE REMPLACEMENT (REGLE 26) 



WOU2/03026 PCT/CHU1/00419 
However, to obtain a correct measurement a reference information is required. So, the position and 
form of the cross f.ex. should be measured for several known (calibration) angles. The number of cali- 
bration angles depend on the complexity of the pattern (its number of degrees of freedom). Each pattern 
has a certain number of degrees of freedom (DOFs) that are necessary to completely define it. More 
degrees 

3.1 Cross pattern * : 

When the metal sheet is being bent, the cross laser pattern changes its position in the field of view 
of the camera as in Fig. 8(a). 

Let us assume that position of the cross center for the bending angles of 180, 120 and 90 degrees 
are 180 degrees calibration point (39), 120 degrees calibration point (39) and 90 degrees calibration 
point (40) respectively and are defined by P m , P m and P 90 . Position of these three points should be 
found prior to any angle measurement and corresponds to the angle calibration. 

Let us denote now by P m the Current point (41) that corresponds to the current measurement of 
the cross center position. The bending angle can then computed from this position by using its relative 
position with respect to the calibration point positions. With the use of a narrow angle objective, the 
radial distortion is low and the trajectory of the cross center during bending can be considered linear. 

To establish the exact position of the P m with respect to P m , P m and P 90 , we project P m onto 
the line P 180 P 90 (cf. Fig. 8(b)) and obtain the point P c . Four points are then mapped to the normalized 
(barycentric) coordinates as shown in Fig. 8 so that P l80 is mapped to 0 and P 9Q to 1. 

3.2 Computation of the angle from three point calibration and cross pattern 

For simplicity, let us consider that laser pattern is a point ray that passes througji the center of the 
cross and we observe the displacement of one point only in the image. In Fig. 9, a side view of the cam- 
era and the sheet is presented. Optical ray of the laser and optical axis of the camera are overlappting. 
Using the definitions of lengths in this figure we first find that: 

*a = %— (4) 
° J2 (tana + I)j2 

where a is the half of the bending angle. Then, in Fig. 10 a top view of the camera and sheet metal 
at different positions is presented where the center of coordinates is the camera optical center and p is 
the angle between the optical axis of the camera and that of the laser. From that setup one can fin± 
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J/ 0 -i> 120 tanP = P' m 
Wo-£) 90 tanp " - P'n 
/f 0 -/) o tanp = i" a 



(5) 



Projecting points P' m , P' m , P'w ^'o on one ^ aB huge plane (49) (that could be selected 
arbitrarily, but parallel to sheet being bent) we then obtain : 



Bo-Dim*** 


= ^180 


tf 0 -£> 120 tanp = 


p D U0 

"On - 

180 


No-D^mp = 


p D 90 


W 0 -i) o tanp = 


P^ 



Eliminating ff Q and tan p we obtain: 



< ^180 ""^120 ^120^ 120~-°18O P l8O 

Use of barycentric coordinates defined as follows (see mapping on Fig. 8.c): 

^120 = ^ > 90 + W > 180~*' P 9o)'l20 
P a = i> 90 + ( /> i80~ /> 90) r a 

the two equations are then reduced to the following pair: 
substitung D a from the equation (4) and optimi2dng, we obtain: 



D o . <«0 '60 



(6) 



(7) 



(8) 



(9) 



(10) 



Eliminating the ratio of unknowns (substituting the first equation into the second) we finally obtain 
the formula for the angle from the coordinate of the cross position 
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To compute the angle one needs to know the positions of calibration points P m , P m , P& and 
that of the current measurement position P m . The barycentric coordinates t 60 , / a of the P 120 and P c 
are then computed with respect to P m and P^. Substituting those barycentric coordinates into the 
previous forumula, gives the tangent of the bending angle. The mapping function position of the pro- 
jected point P c would then be mapped to the bending angle according to the function in Fig. 1 1(b). 

This method requires three calibration angles 180° , 120° , 90° . Using the orientation of the cross, 
the number of calibration points might be reduced to one. All the stages of the angle computation from 
cross position are presented in Fig. 19. 

3.3 Computation of the angle from several parallel lines 

Taking a more complex pattern as a set of parallel lines allows to have more degrees of freedom 
(DOFs) per measurement and thus less calibration angles. For a pattern of more than 5 parallel lines the 
only calibration that is required is the 180 angle. Moreover, the position of the calibration plane is not 
constrained to the bending matrix and can occur anywhere on the optical path of the camera. 

IV. BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 Side view of one module. 

Fig. 14 Configuration where two modules are placed above the sheet metal being bent. 
Fig. 12 Configuration where two modules are placed below the sheet metal being bent 
Fig. 15 Configuration where two modules are placed above the sheet metal being bent. Mirrors are used 

to mount modules closer to the bending ram. 
Fig. 16 External connections of two modules. 
Fig. 2 General algorithm of bending angle calculation. 
Fig. 3 Algorithm for localization of the "cross" pattern. 

Fig. 4 The stage of the fast cross pattern localization corresponding to acquiring scan lines and finding 
three pairs of points, (a) Example of the profile. (b,c,d) Three possible configurations. 

Fig. 5 Second stage of localization where pairs of points are sorted and attributed to two lines accord- 
ing to their collinearity. 

Fig. 7 Computing precise position of cross pattern, (a) Acquisition of the short scan lines in pattern 
neighbourhood, (b) Maxima detection, (c) Subpixel maxima detection, (d) Fitting lines to the 
detected points and the definition of the intersection. 
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Fig. 8 Cross displacement during bending, (a) Calibration positions of the cross, (b) Computation of 
the relative cross center position, (b) Mapping crosss position to relative (barycentric) coordi- 
nates. 

Fig. 9 Side view of viewing geometry. 

Fig. 1 1 Mapping between position of the cross point and the (half) bending angle. 
Fig. 1 8 Algorithm for calibrating the system. 

V. BEST MODE FOR CARRYING OUT THE INVENTION 

This section, describes the implementation of the described invention as it was realized in the 
Smart Bending Angle Calculator SBAC-1 135. 

L System architecture 

The camera and laser can be mounted in one single casing as shown in Fig. 1 . Each module has the 
"all-in-one" casing structure that is illustrated on Fig. 1. The main casing contains the Motherboard 
(23) with the processing unit and communication interfaces connected to External connectors (36), the 
VGA board (24). The Sensor controller (26) is connected by a Flexible connector (46) to the Sensor 
board (51) hosting the CMOS sensor (50). The camera Objective (33) is mounted on this board The 
DAC Card (25) is connected to the Laser (7) and can control its power and intensity through the Laser 
power cable (27) and Laser intensity control cable (28). 

External connectors of one module are presented on Fig. 16. The master module contains RS232 
port (10), RS485 port (11), Digital port (12), VGA port (14), Master network or CAN-bus connector 
(3 8). The slave module has only. ; 

2. Flexible geometric setup 

To have a precise measurement of the the bending angle, one need to make measurements on both 
sides of the press. This requires the use of two modules (master and slave) on both sides of the press 
that are simultaneously measuring the angle and communicating between themselves. The mounting of 
the modules can be performed in several ways shown in Fig. 12, Fig. 13, Fig. 14 and in Fig. 15. In the 
first "standard" configuration, two modules Master module (5) and Slave module (6) are positioned 
below the metal sheet being bent on both sides of the Bending ram (2) and pointing upward to the 
Bended metal sheet (3). The number of master/slave pairs (that measure the angle) along the benging 
ram can be more than one. 

None of the modules require a specific position as long as the pattern is visible at the starting bend- 
ing angle and at the ending angle. Therefore, modules can be positioned with respect to the sheet being 
bent according to the particular requirements of the user. 
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3. Fast pattern localization 

The fast pattern localization method described in Section III can be efficiently implemented on a 
CMOS sensor that allows pixel-based access. A pixel-based image sensor is an image sensor which 
allows for accessing each individual pixel in any chronological or spatial order at any point in time. 
Each module is equipped with CMOS sensor and with a controller that allows a random-pixel access. 

Moreover, the sensor can be used in a special mode that gives priority to the horizontal direction of 
sensor access. Doing so, the acquisition of horizontal line segments is even faster which suits perfectly 
the scanline approach. 

4. Calibration 

The calibration operation is performed according to the algorithm described in Fig. 18. One needs 
to provide the sheet metal in the correct angle orientation with respect to the bending matrix as shown 
in Fig. 17. This is achieved with specially designed calibration tools for 180° , 120° and 90* degrees. 
Their three-point contact provides an easy use. 

5. Communication 

Two modules are connected by a RS485 or fieldbus interface and the computed angle and other 
commands are communicated between them over a specially designed protocol. 

The resulting angle is then communicated by the master module to the device that controls the 
bending machine over the RS232 or fieldbus interface with proprietary protocol or standard. 



;.'.9 



FEUILLE DE RE REPLACEMENT (REGLE 26) 



WO 02/03026 



PCT/CH01/00419 



VL WE CLAIM: 

1. An orientation measurement apparatus and method comprising a high-dynamic sensor capable of 
reading single pixels, lenses, a processing unit, communication interfaces and a means for projecting 
a light pattern onto the surface of a planar object. The said apparatus can be integrated in a single 
casing. The light pattern is a grid of more than two crossing lines allowing for the unconstrained 
measurement of the surface orientation of the planar object in the three-dimensional space. 

2. An orientation measurement apparatus and method as in claim 1 for the measurement of the orienta- 
tion of sheet metal in metal processing and metal bending applications. 

3. An orientation measurement apparatus as in claim 1, whereas the means for projecting a light pat- 
tern is a laser with a pattern generator. 

4. An orientation measurement apparatus as in claim 1, whereas the light pattern is a set of more than 
two parallel lines, allowing for unconstrained measure of the surface orientation of the planar object 
in the three-dimensional space. 

5. An orientation measurement apparatus as in claim 1 , whereas the light pattern is a circle, an ellipse, 
a set of circles or ellipses, a matrix of dots allowing for unconstrained measure of the surface orien- 
tation of the planar object in the three-dimensional space. 

6. An angle measurement method as in claim 1, whereas a line-based access of the sensor is used to 
find the position of the light pattern in the image. Several lines are acquired by the image sensor to 
find the intersection of the intensity maxima of the pattern with the scanning lines. The search lines 
can be horizontal, vertical, in non-consecutive order or diagonal with arbitrary angles. 

7. An angle measurement method as in claim I, whereas a pixel-based access of the sensor is used to 
find the position of the projected light patterns in the image. Intensity maxima of the projected light 
patterns are found using pixels that are accessed in a non-consecutive and non-continuous way. 

8. The angle measurement system as in claims 1 comprising two or more angle measurement appara- 
tuses providing the possibility of exchanging data. 

9. The angle measurement system as in claims 1 that needs less than three reference angle measure- 
ments to be fully calibrated if the cross pattern is used. In the case of the pattern of more than five 
parallel lines, only one reference angle measurement is required for calibration. The module position 

1 ■. , 
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and orientation with respect to the bending ram in the latter case is not constrained. 

10. The angle measurement system as in claims 1 that uses a calibration tools with three points of con- 
tact with the matrix of the bending machine. 

11. The angle measurement system as in claims 1 whereas the light pattern is detected by computing the 
difference between an image taken with the light pattern projected onto the planar object and 
another image taken without the light pattern projected onto the planar object. 
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Fig. 14 
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Fig. 15 
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Fig. 16 
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